{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Juliabox is based on JuliaRun\n",
    "\n",
    "The distributed compute facilities in JuliaBox are provided by [JuliaRun](https://juliacomputing.com/products/juliarun.html), Julia Computing's solution for scaling Julia processes. There are available in JuliaBox via the `JuliaRunClient` package, which allows you to run interactive or batch distributed processes.\n",
    "\n",
    "For the free edition of JuliaBox, users are limited to 2 CPU cores. For running larger clusters, please contact `juliabox@juliacomputing.com`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Understanding the cluster\n",
    "\n",
    "`JuliaRunClient` provides a `Context` object which is the entry point into the cluster. All operations need a reference to this object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "using JuliaRunClient\n",
    "ctx = Context()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see if we can connect to the cluster"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Running batch job \n",
    "\n",
    "JuliaRun is well suited to run batch processes in a distributed environment. We have below an example of running the Monte Carlo Pi computation as batch job. \n",
    "\n",
    "First, we create an instance of a `JuliaParBatch` object, with name _mcpi_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "job = JuliaParBatch(\"mcpi\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We then _submit_ the job to the *JuliaRun* system, providing it with the scripts to run on the master and worker nodes, as well as the attached filesystem volume names. Update `num_workers`, `master_cpu`, `master_mem`,`worker_cpu` and `worker_memory`  as appropriate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_workers=2\n",
    "master_cpu=1\n",
    "master_mem=\"2Gi\"\n",
    "worker_cpu=1\n",
    "worker_memory=\"2Gi\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@result submitJob(ctx, job; start_script=\"/juliabox/scripts/mcpi_pmap_master.jl\",\n",
    "        run_volume=\"juliabox\", image=\"juliabox\", nworkers=num_workers,\n",
    "        cpu=master_cpu, memory=master_mem,\n",
    "        shell=\"/juliabox/scripts/master.sh\", worker_shell=\"/juliabox/scripts/worker.sh\",\n",
    "        worker_cpu=worker_cpu, worker_memory=worker_memory,\n",
    "        worker_start_script=\"/juliabox/scripts/mcpi_pmap_worker.jl\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Wait for a while\n",
    "sleep(25)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While the job is running, we can query it's status. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@result getJobStatus(ctx, job)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The logs for the job is visible on the shared filesystem"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Wait for a while to see the output\n",
    "sleep(20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    ";tail /mnt/juliabox/logs/output1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the job is completed, we can clean it up and free it's resources. Again, this is important to do for cost reasons. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@result deleteJob(ctx, job; force=true)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    ";rm -f /mnt/juliabox/logs/output1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Julia 0.6.0",
   "language": "julia",
   "name": "julia-0.6"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "0.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
